How NVIDIA Uses AI For Performance Capture

With NVIDIA's newest artificial intelligence (AI) technology, anyone can capture a performance and translate it into an animation or use it on an AI avatar or image more efficiently without bulky and expensive hardware. AI is quickly becoming a mainstay in the world, and it continues to improve and become increasingly more powerful. As AI technology gets better and better, AI opens the door to a plethora of opportunities for people.

Motion capture has been around for hundreds of years since animators used a technique known as rotoscoping to paint over a live-action video to make more life-like animation. As technology improved, motion-capture equipment was released to allow actors to have their performances captured and translated into animations or CGI. Some notable examples of movies using this technology are Rise of the Planet of the Apes, Avatar, Thanos from Avengers: Infinity War, and others.

Related: What Is The Future Of Dall-E AI Technology?

While using motion capture suits to capture the performance of the actor almost perfectly, the drawback of this technology is it costs thousands of dollars, much more expensive than a software solution. With NVIDIA Vid2Vid Cameo, facial expressions and body movements are tracked effectively with AI technology. The AI was trained on 180,000 videos, and by using 20 key points on a face, the AI can encode those points onto a digital character, picture, painting, or any other image of a person. This is done in real-time and can be done with standard 2D video recordings.

With this AI technology from NVIDIA, animators can easily and quickly see how a specific movement will look in animation to help speed up their workflow. The technology will also allow for a cheaper method of achieving good performance capture for amateur filmmakers or those who want to animate their own virtual 3D characters. Of course, the better the camera systems, lighting and other hardware technology that someone has access to will make the AI performance capture look better, but NVIDIA's AI advancements are extremely impressive.

Many think AI may be on the verge of taking away jobs from artists with technology such as Dall-E or other AI art generators. And now, with AI able to animate people with motion-capture without the need for expensive hardware, is this going to put animators out of a job? The answer, as always, is no. Instead, this technology will become a valuable tool for animators and directors by speeding up their workflow and making their lives easier NVIDIA's AI technology should be viewed as a tool to make people's lives easier and not as a technology to be feared.

Source: NVIDIA